Generally, when using a video camera or a digital still camera to photograph a color image, the incident light passes through filters for extracting certain wavelength components such as the basic color components R (red), G (green) and B (blue). In two-dimensional imaging, the imaging unit is composed of many pixels arranged in the vertical and horizontal directions. Each pixel of the two-dimensional image contain either red, green or blue color light because of the filtering of the incident light. A color sensing array of this type is disclosed in the document U.S. Pat. No. 3,971,065 (B. E. Bayer).
According to one of several alternative techniques, the type of filter is changed for every pixel and the filters are cyclically aligned in the order: G, R, G, R . . . in the even rows of the pixel array of the sensor, and B, G, B, G . . . in the odd rows of the pixel array of the sensor. As a consequence, information of the photographed colored object is obtained only once every three pixels. In other words, an object cannot be color photographed other than in units of three pixels.
To reconstruct all the pixels of the two-dimensional image of the photographed object, it is necessary to interpolate chrominance pixel data to obtain the color components of red, green and blue color using information contained in neighboring pixels of the pixel to be reconstructed/enhanced. Generally, a value corresponding to the interpolated pixel is reconstructed by averaging corresponding values of a plurality of pixels surrounding the location of the pixel to be interpolated. Alternatively, the interpolated pixel may be determined by averaging the values of the pixels remaining after discarding pixels of maximum and minimum values of the neighbor pixels of the pixel to be interpolated. Also well known are techniques for detecting an edge of a photographical object by analyzing the pixels surrounding the considered cluster.
U.S. Pat. No. 5,373,322, U.S. Pat. No. 5,053,861, U.S. Pat. No. 5,040,064, U.S. Pat. No. 6,642,962, U.S. Pat. No. 6,570,616, U.S. published Patent Application No. 2003/0053687, U.S. Published Patent Application No. 2003/0007082, U.S. published Patent Application No. 2002/0101524, U.S. Pat. No. 6,366,694, European Patent Publication No. 0 497 493, European Patent Publication No. 1 176 550, and European Patent Publication No. 1 406 447, disclose techniques that are used in image processing.
Generally, the data acquired by the sensor according to a special pattern, for example the one known in the art as Bayer color-filter array (CFA), a pattern of which is characterized by associating just one of the three basic color components to each pixel, therefore a good quality RGB image is obtained by a specific image processing sequence (via hardware or image generation pipeline (IGP) or via software) to generate a high quality color image. Generally, in cascade of such an image processing subsystem is associated a data compressing block for reducing the band necessary for transmitting the color reconstructed image from the image processing subsystem or a mass storage support or to a remote receiver or to a display unit.
Data, for example in Bayer format, as acquired by a digital sensor clearly represent a gross approximation of the chromatic components of the reproduced scene, and it is of a paramount importance the accuracy with which color reconstruction via interpolation algorithms is performed on the raw chrominance data acquired by the digital sensor. Usefulness of Bayer data compression has emerged quite recently. Since it requires a relatively inexpensive solution, both in terms of computational complexity and hardware (HW) requirements, the most common way to address the problem has been to split Bayer image color channels and compress them independently using an efficient compression algorithm, for example differential pulse code modulation (DPCM), e.g. as discussed in R. M. Gray, D. L. Neuhoff, “Quantization”, IEEE Trans. on Information Theory, vol. 44, n. 6, October 1998.
Acharya et al. (U.S. Pat. No. 6,154,493) have proposed a rather sophisticated compression method for images in Bayer pattern format. According to their approach the Bayer pattern image is considered as containing four independent color planes. In fact, since there are twice as many green related pixels as either of blue or red pixels, two distinct green planes are constructed mainly: G1 (containing green pixels in the same row as red pixels) and G2 (containing green pixels in the same row as blue). B and R color planes represent blue and red pixels respectively.
To exploit advantageously both the correlation between an R associated pixel and its G1 associated neighboring pixels and the correlation between associated pixel and its G2 and B associated neighboring pixels, compression is performed distinctly on each plane. G1 and G2 associated pixels are compressed directly, while each R pixel value is subtracted by its “west” neighboring G1 pixel value. Likewise, the difference (B−G2) is computed and planes (R−G1) and (B−G2) are then compressed. Compression is obtained in two main steps. Firstly, a 2-dimensional Discrete Wavelet Transform (DWT) is applied and secondly, DWT coefficients are quantized. DWT data are used because make possible to describe abrupt changes better then Fourier transform data.
The result is a lossy compression that is perceived by Human Visual System (HVS) to be lossless when decompressing is done. Decoding consists simply on inverting the coding steps: data are dequantized and then the Inverse DWT (IDWT) is performed. The four color channels may be separately decompressed. Once IDWT is performed, by adding back G1 to (R−G1) recovered value and G2 to (B−G2) recovered value, each Bayer original pixel value is restored.
Another sub band-coding compression method is described in T. Toi, M. Ohita, “A Sub band Coding Technique for Image Compression in Single CCD Cameras with Bayer Color Filter Arrays”, IEEE Transaction on Consumer Electronics, Vol. 45, N. 1, pp. 176-180, February 1999.
Lee and Ortega proposed an algorithm for Bayer image compression based on Jpeg as discussed in Sang-Yong Lee, A. Ortega, “A Novel Approach of Image Compression in Digital Cameras With a Bayer Color Filter Array”, In Proceedings of ICIP 2001—International Conference on Image Processing—Vol. III 482-485—Thessaloniki, Greece, October 2001.
Most digital cameras yield full color image by compressing an IGP image into a Jpeg image, after the color interpolation process performed by the IGP pipeline, but in this way in the interpolation step redundancy is increased, before being reduced by Jpeg compression. To overcome this drawback, the algorithm performs an image transformation to encode the image as a Jpeg image before color interpolation. The algorithm includes three basic steps. A pre-processing of the image to convert Bayer data to YCbCr format with 4:2:2 or 4:2:0 sub sampling. The size of color image is, of course, three times bigger than that the Bayer data and, to avoid increasing redundancy, the size of data should not be increased after color format conversion. Thus, conversion is done as follows: each Y data includes common blue and red pixels and a different green pixel, while Cb and Cr data includes blue, red and the average of two properly chosen green pixels. After this transformation, Y data presents blank pixels, therefore Jpeg compression cannot be directly applied. Therefore, the second step is another transformation that simply rotates the Y data 45°.
After rotation, Y data are concentrated at the center of the image of rhombus shape by removing rows and columns that contain blank pixels. Finally, Jpeg compression is performed. Blocks located along the boundaries of Y image data are filled by using a mirroring method. Other Bayer data compression techniques are described in U.S. Pat. No. 5,172,227 to Tsai et al. and entitled “Image Compression with Color Interpolation for a Single Sensor Image System” and Le Gall, A. Tabatabai, Subband Coding of Digital Images Using Symmetric Shor Kernel Filters and Arithmetic Coding Techniques, in Proceedings of the ICASSP 88 Conference, (New York), pp. 761-764, April 1988.
Vitali A., Della Torre L., Battiato S., Buemi A. in “Video and Image Lossy De/Compression By Perceptual Vector Quantization”, EP-A-1406447 disclose a compression technique based on Vector Quantization wherein the quantization step varies taking into account certain subjective quality sensibility characteristics of the Human Visual System. Bayer pattern values are gathered according to the channel they belong to in groups of two pixels and the so generated couple is quantized with a function that accounts for the effects of “Edge Masking” and “Luma Masking”.
Modestino et al. describe in “Adaptive Entropy-Coded Predictive Vector Quantization of Images” IEEE Transaction on Signal Processing, Vol 40 No 3, March 1992 discuss two dimensional predictive vector quantization of images subject to an entropy restraint.
Although the source data format of a Bayer pattern is amenable to compression with relatively simple means and with a relatively low computational burden, the known image compression algorithms so far proposed require a non-negligible computational complexity and a relatively large external memory requirement.